Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Response Ops][Task Manager] Add new configurable values #190934

Merged
merged 6 commits into from
Aug 26, 2024

Conversation

ymao1
Copy link
Contributor

@ymao1 ymao1 commented Aug 20, 2024

Resolves #190734

Summary

Making the following configurable:

  • discovery interval for new discovery service
  • lookback time when querying for active nodes in discovery service
  • kibanas per partition for new partitioning functionality

@ymao1 ymao1 changed the title Tm configs [Response Ops][Task Manager] Add new configurable values Aug 21, 2024
@@ -67,9 +73,17 @@ const requestTimeoutsConfig = schema.object({

export const configSchema = schema.object(
{
active_nodes_lookback: schema.number({
defaultValue: DEFAULT_ACTIVE_NODES_LOOK_BACK_S,
min: 10,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if we should set a max as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would 5 minutes make sense? If a user sets 5m, it would take 5m for a node that crashed to disappear from the list of active nodes. Perhaps its a sufficient high bound that we'd want to support?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 730d933

allow_reading_invalid_state: schema.boolean({ defaultValue: true }),
/* The number of normal cost tasks that this Kibana instance will run simultaneously */
capacity: schema.maybe(schema.number({ min: MIN_CAPACITY, max: MAX_CAPACITY })),
discovery_interval: schema.number({
defaultValue: DEFAULT_DISCOVERY_INTERVAL_MS,
min: DEFAULT_DISCOVERY_INTERVAL_MS,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if we should set a max as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could see 1 or 5 minutes as the max, we could allow a smaller min, maybe 1s or so?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 730d933 to set max of 5 minutes

@ymao1 ymao1 self-assigned this Aug 21, 2024
@ymao1 ymao1 added release_note:skip Skip the PR/issue when compiling release notes Feature:Task Manager Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v8.16.0 labels Aug 21, 2024
@ymao1 ymao1 marked this pull request as ready for review August 21, 2024 17:01
@ymao1 ymao1 requested review from a team as code owners August 21, 2024 17:01
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would putting these new discovery configs under a name space (like discovery.) be better?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 730d933

@@ -32,6 +32,12 @@ export const DEFAULT_WORKER_UTILIZATION_RUNNING_AVERAGE_WINDOW = 5;
export const CLAIM_STRATEGY_UPDATE_BY_QUERY = 'update_by_query';
export const CLAIM_STRATEGY_MGET = 'mget';

export const DEFAULT_DISCOVERY_INTERVAL_MS = 1000 * 10;

export const DEFAULT_ACTIVE_NODES_LOOK_BACK_S = 30;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe duration format like 1s 1h. we already have validateDurationSchema

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 730d933

Copy link
Contributor

@ersin-erdal ersin-erdal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Left two notes for nice-to-have. Not a blocker.

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @ymao1

@ymao1 ymao1 merged commit 1ee45da into elastic:main Aug 26, 2024
36 checks passed
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Aug 26, 2024
@ymao1 ymao1 deleted the tm-configs branch August 26, 2024 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting Feature:Task Manager release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v8.16.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make some new task manager variables configurable via kibana.yml
7 participants